Automated feature selection is important for text categorization to reducethe feature size and to speed up the learning process of classifiers. In thispaper, we present a novel and efficient feature selection framework based onthe Information Theory, which aims to rank the features with theirdiscriminative capacity for classification. We first revisit two informationmeasures: Kullback-Leibler divergence and Jeffreys divergence for binaryhypothesis testing, and analyze their asymptotic properties relating to type Iand type II errors of a Bayesian classifier. We then introduce a new divergencemeasure, called Jeffreys-Multi-Hypothesis (JMH) divergence, to measuremulti-distribution divergence for multi-class classification. Based on theJMH-divergence, we develop two efficient feature selection methods, termedmaximum discrimination ($MD$) and $MD-\chi^2$ methods, for text categorization.The promising results of extensive experiments demonstrate the effectiveness ofthe proposed approaches.
展开▼
机译：自动特征选择对于文本分类对于减小特征尺寸并加快分类器的学习过程非常重要。在本文中，我们提出了一种基于信息论的新颖而有效的特征选择框架，旨在以特征的分类能力对特征进行排序。我们首先回顾两个信息量度：用于二元假设检验的Kullback-Leibler散度和Jeffreys散度，并分析它们与贝叶斯分类器的I型和II型误差有关的渐近性质。然后，我们引入一种称为Jeffreys-Multi-Hypothesis（JMH）散度的新散度度量，以测量用于多类分类的多重分布散度。基于JMH散度，我们开发了两种有效的特征选择方法，即最大区分度（$ MD $）和$ MD- \ chi ^ 2 $方法，用于文本分类。大量实验的有希望的结果证明了该方法的有效性。
展开▼
